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As  mentioned  in  the  attached  report,  we  have  leveraged  the  results  of  our  work  on  a  neurally  motivated 
model  of  reflexive  reasoning  to  design  a  real-time  large-scale  knowledge  based  reasoning  system.  We  have 
completed  the  implementation  of  a  system  on  the  CM-5  and  tested  it  with  (i)  large  randomly  generated 
knowledge  bases  and  (ii)  WordNet,  a  lexical  database  containing  knowledge  about  hypernym,  hyponym, 
synonym,  and  antonym  relationships  between  lexical  items.  In  the  first  case,  the  system  responds  to  retrieval 
queries  in  under  10  msec,  and  inferential  queries  of  upto  depth  5  in  under  200  msec.  In  the  second  case,  the 
system  exhibits  a  speed-up  of  5-450  compared  to  a  traditional  serial  implementation  of  WordNet.  I  remain 
interested  in  making  the  system  available  for  applications,  of  potential  interest  to  the  military,  that  might 
benefit  from  the  capabilities  of  this  efficient  large-scale  knowledge-based  system. 
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Progress  Summary 

The  results  of  our  work  on  a  biologically  motivated  model  of  reflexive  reasoning  were  used  to  implement  a 
large  scale  system  for  performing  rapid  reasoning  using^very  large  knowledge  bases.  The  system  —  SHRUTI- 
CM-5  runs  on  a  32  node  CM-5,  and  has  been  tested  on  a  large  randomly  generated  knowledge  base  containing 
over  500,000  items  and  WordNet,  a  real-world  lexical  database.  The  system’s  response  time  is  fast  enough 
to  support  retrieval  and  inferencing  for  real-time  applications  such  as  speech  understanding.  We  have  also 
made  some  progress  in  addressing  the  catastrophic  interference  problem. 

Progress  Report 

We  have  continued  our  investigation  of  the  representational  capabilities  of  spatio-temporal  networks  and 
their  application  to  reflexive  reasoning.  We  have  leveraged  the  results  of  our  reflexive  reasoning  system 
based  on  temporal  synchrony  to  build  a  system  for  performing  rapid  reasoning  using  very  large  knowledge 
bases.  The  system’s  response  time  is  fast  enough  to  support  inferencing  for  real-time  applications  such  as 
speech  understanding.  The  system  can  respond  to  retrieval  as  well  as  inferential  queries  within  a  few  hundred 
milliseconds.  The  system  running  on  a  32  node  CM-5  has  been  tested  on  (i)  a  randomly  generated  knowledge 
base  of  over  500,000  rules,  facts,  and  types  and  (ii)  WordNet,  a  lexical  database  containing  knowledge  about 
hypernym,  hyponym,  sjmonym,  and  antonym  relationships  between  lexical  items.  In  the  first  case,  the 
system  responds  to  retrieval  queries  in  a  few  msec,  and  inferential  queries  of  up  to  depth  5  in  under  200 
msec.  In  the  second  case,  the  system  exhibits  a  speed-up  of  5-450  compared  to  the  serial  implementation. 
This  work  will  lead  to  the  PhD  dissertation  of  D.R.  Mani  (expected  date  of  completion,  August  1995). 

As  reported  previously,  we  are  investigating  a  possible  solution  to  the  “catastrophic  interference  problem”. 
This  problem  is  perhaps  the  biggest  impediment  in  the  development  of  scalable  learning  systems.  The  solution 
under  investigation  is  as  follows:  Initially  the  system  focuses  on  a  small  number  of  categories.  After  it  learns 
these  categories,  it  tries  to  identify  which  features  formed  in  the  “hidden  layer”  play  a  crucial  role  in  the 
recognition  of  these  categories.  The  system  “freezes”  these  crucial  features  by  fixing  their  input  weights  and 
as  a  result,  they  cannot  be  obliterated  by  subsequent  learning  (although  the  features  may  tmdergo  some  fine 
tuning).  These  frozen  features  however,  are  available  for  solving  other  tasks  and  can  be  a  part  of  structures 
learned  subsequently  to  recognize  other  categories.  Our  hope  is  that  the  set  of  features  will  gradually  stabilize 
and  eventually,  learning  a  new  category  will  primarily  consist  of  combining  existing  features  in  novel  ways. 
These  ideas  are  being  investigated  in  the  context  of  training  spatio-temporal  networks  to  recognize  digit 
strings.  Our  efforts  thus  far  have  been  moderately  successful.  A  strength  of  learning  algorithms  such  as 
backpropagation  is  that  they  produce  “distributed”  and  highly  redundant  representations.  This  attribute 
however,  becomes  a  drawback  in  the  context  of  our  approach  because  it  makes  it  difficult  to  isolate  a  compact 
set  of  critical  features.  Our  current  effort  is  focused  on  (i)  identifying  a  suitable  objective  function  that  would 
prompt  the  hidden  layer  to  learn  relatively  compact  representations  and  ii)  developing  a  robust  but  tractable 
criteria  for  identifying  a  set  of  critical  features  (the  problem  of  identifying  an  optimal  set  of  critical  features 
is  intractable  since  doing  so  requires  the  evaluation  of  all  2^  possible  subsets  of  n  hidden  nodes).  We  are 
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Refereed  Journal  Paper  accepted 

1.  “Recognizing  hand-printed  digit  strings  using  a  modular  spatio-temporal  connectionist  network”,  L. 
Shastri  and  T.  Fontaine.  To  appear  in  Connection  Science. 

Book  Chapter  Published 

1.  “Reasoning  with  rules  and  variables  in  neural  nets”,  V.  Ajjanagadde  and  L.  Shastri.  In  Intelligent 
Hybrid  Systems,  S.  Goonatilake  and  S.  Kebbal  (Eds).  John  Wiley  &  Sons.  1995. 

Book  Chapter  in  Press 

1.  “Structured  Connectionist  Model”,  L.  Shastri.  In  The  Handbook  of  Brain  Theory  and  Neural  Networks, 
M.  Arbib  (Ed),  MIT  Press.  In  Press.  J 

Invited  Colloquia 

1.  Joint  Philosophy,  Neuroscience  and  Psychology  Colloquium,  Washington  University  St.  Louis.  Oct. 

1994. 

2.  AI  and  Robotics  Seminar,  Division  of  Computer  Science,  University  of  California  at  Berkeley.  March. 

1995. 


Contributed  Presentation 

1.  “A  spatio-temporal  approach  to  visual  pattern  recognition”  in  the  workshop  on  ”  Neural  Network  Ar¬ 
chitectures  with  Time  Delay  Connections  for  Nonlinear  Signal  Processing:  Theory  and  Applications” . 
NIPS*94,  Vail  Colorado.  Dec.  1994. 
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